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Abstract 

A Helmet Mounted Display system has been developed. It provides the 
capability to display stereo images with the viewpoint tied to subjects head 
orientation. This type of display might be useful in a telerobotic environment 
provided the correct operating parameters are known. The effects of update 
frequency were tested using a 3D tracking task. The effects of blur were 
tested using both tracking and pick-and-place tasks. For both we found that 
operator performance can be degraded if the correct parameters are not used. 
We are also using the display to explore the use of head movements as part 
of gaze as subjects search their visual field for target objects. 

Introduction 

Stereo displays are now commonly used as a visual interface for telero- 
botic systems because they provide better telepresence than monoscopic 
displays [1],[3],[13]. The hardware has improved considerably since stereo 
displays have been used, but our knowledge of the perceptual aspects of using 
stereo display systems remains very limited. The video images from a remote 
site could be degraded by a number of factors such as blur, noise, contrast or 
illumination changes that could differentially affect performance with stereo 
and non-stereo displays. The effects of these degradations on performance 
has not been closely examined except in the case of contrast [4]. Without this 
knowledge, stereo displays might be used in inappropriate situations, i.e. 
where an unnecessary increase in performance is offset by increases in cost, 
transmission bandwidth or system complexity [5]. 

In order to test the usefulness of our Helmet Mounted Display system, 
we decided to explore how changing the parameters of the display affected 
the performance of some representative telerobotic tasks. Previous studies 
have shown that increasing the apparent interocular distance of the display 
from zero, decreases mean completion time for a pick and place task.[l] 
Here we explore the effects of different update frequencies and blur on per- 
formance. 

The Helmet Mounted Display 

Helmet mounted displays were designed as a more comfortable inter- 
face that provides the human operator with telepresence, i.e. the feeling that 
he/she is in the remote environment and controlling the manipulator directly. 
The helmet is equipped with a stereo display and can detect the operator’s 
head motions in order to move the remote stereo cameras in a similar fashion. 
Our experiments simulated the remote environment with computer graphics 
and standard stereoscopic display formulas [1]. 



Figure 1. Helmet Mounted Display This system provides a stereo view 
through two video camera viewfinders, one for each eye. Head orientation is 
measured with the Helmholtz coil system, (outer coils) 


Two degrees-of-freedom are detected by the head orientation sensing 
device, horizontal and vertical rotations. The sensing system is comprised of 
two magnetic Helmholtz coils with normal magnetic fields and a sensing coil 
mounted on the inside top of the helmet. Each field coil generates a different 
Sine wave magnetic flux (50kHz for horizontal, 75kHz for vertical). The 
induced flux in the sensing coil is then amplified and separated into the 
respective components. The magnitude of each component is dependent on 
the head orientation with respect to the corresponding field coil [2]. 



Figure 2. Robot Manipuation Task. Stereo images and head coordinated 
views give operators a sense of telepresense. The relationship between robots 
and objects might be more easily seen in such a display. 


The stereo display is comprised of two 1-inch viewfinders (Sony VF- 
208) mounted on the helmet by means of a 5 degree-of-freedom frame. The 
frame has three orthogonal slidings and two rotations. Each viewfinder also 
has a converging lens which forms the virtual image of the stereo pair 25cm 
behind the actual display screen. The sixfold magnification of the lens makes 
the display screen look like a 6- inch screen (33 degrees wide). When 
adjusted appropriately, most observers can fuse the two images into a single 
three-dimensional image. A 0.5 kg counterweight was fastened to the back of 
the helmet to counterbalance the viewfinders. 

Update Frequency Requirements 

One important goal of using a HMD system is to provide the user with 
a stronger sense of telepresence from the enforced correlation of head move- 
ments and displayed view direction. If the scene is not updated rapidly 
enough this correlation (called Space Constancy in the realm of natural 
human gaze movements and psychophysics [15]) might be lost, but in a 
telerobotic setting, limitations in hardware or in signal bandwidth might limit 
the update speed, We suspect that there is some band of update frequencies 
above which there is no noticeable change in the display quality but below 
which operator performance falls to unacceptable levels. It is this range 
where performance is affected by update frequency that we explore here. 

We tested the effects of update frequency by using a 3D tracking task. 
Figure 3 shows a sample display from the experiment. The subjects viewed a 
virtual space from above the edge of a ground grid. In the space was a box 
target and a spherical cursor. The cursor was controlled through two joys- 
ticks. The subjects were asked to keep the sphere in the box. The target 
moved in a pseudorandom pattern in three dimensions. The target trajectory 
was a sum of II sinusoids with frequencies from 0.007 Hz to 5.8 Hz. The 
amplitudes simulated a low pass filter with a corner frequency of about 0.08 
Hz. The field of view of the scene extended to 60 degrees so that when the 
target was at the extremes of its motion, head movements were performed to 
keep the target in the visual field. Under these conditions each subject car- 
ried out the task for 10 update frequencies ranging from 15 Hz to 0.5 Hz 
presented in a random order. 
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Figure 3. 3D Tracking Tasks. The subject tries to keep the ball inside the 
box that continually moves in a pseudorandom pattern. The ball is controlled 
by two joysticks. 


Tracking mnd iMd Scaponn 




Due to the high frequency content of the input signal, the normalized 
rms error is high even for the fastest update frequency. We normalized the 
rms error to the situation where there was no operator input Therefore, error 
values greater than one indicate tracking worse than doing nothing at all. 
This situation occurs for the lowest group of update frequencies that we 
tested. Figure 4 shows the rms error for all the update rates tested. 

We measured both the cursor position and the head movements even 
though the subjects were not told to track the target with their head. Figure 5 
(top and middle) shows the response of two subjects at the highest update fre- 
quency for both tracking and head movements. It shows that the bandwidth 
of the head movements is smaller than that of the tracking. This is not 
unexpected since the subjects were only using head movements to assist in 
the tracking. However, the bottom panel, which also shows the response at 
the lowest update frequency, indicates that manual tracking degrades to the 
same bandwidth as head movements. Even so, figure 6 shows that at a par- 
ticular input frequency, around the comer frequency of the tracking at the 
highest update rate, the manual tracking is better than head movements 
throughout the experiment. In general we discovered the not surprising result 
that if the image doesn’t change with head movement, subjects tend not to 
move their heads. But, they seem to try to track the target even after the 
update rate slows considerably. 


Normalized RMS Error 



Update Rate (Hr) 

Figure 4. Tracking Performance. The rms error is normalized to the case of 
no response. As the update frequency falls the rms error rises; at the lowest 
frequencies the tracking is worse than no response. 
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nglirt 5. Tracking Response. Two subjects (top and middle panels) show 
th« the tracking m has a higher bandwidth than the head movements [H] at 
the highest update rate. At the lowest update rate (bottom panel) tracking ft] 
and head movements pi] have the same bandwidth. 


Figure 6. Tracking Gains. The gains of the tracking and head movements 
for an input frequency of 0.75Hz across all update frequencies shows how the 
gains fall as update frequency goes down. It also shows that tracking is more 
accurate than head movements, this is because the subjects were not asked to 
track the target with their heads. 
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The Effects of Blur on Task Performance 

Psychophysical experiments can provide a basis for predicting perfor- 
mance changes under these degrading conditions. For example, experiments 
studying the effect of blur on human stereoacuity have shown that target blur 
reduces stereoacuity. Most of these experiments used blur induced with spec- 
tacle lenses and measured the stereoacuity with a depth -discrimination test 
such as the Howard-Dolman lest [6]-[ 9]. Blur caused by spatial frequency 
filtering [8] has also been shown to cause similar degradations in stereoacuity. 
In addition, it was found that blur of only one eye resulted in a greater deficit 
than equal amounts of blur in both eyes. Figure 7 (top) shows some of the 
results from Lit’s and Westheimer and McKee’s experiments with monocular 
and binocular spectacle blur. The data for binocular blur is shown by the 
solid plot symbols. Stereo displays differ from monoscopic displays in that 
they also use the disparity cue to display depth. Therefore, we conclude that 
stereoacuity, or the ability to discriminate two points in depth, is important in 
telemanipulation. Furthermore, we would expect operator performance to 
d ecr ease as the one or both of the Images were increasingly blurred. 


To confirm this hypothesis, we examined the effect of visual target blur 
on the performance of simulated telerobotic tasks. Using the helmet mounted 
display described previously, two generic telerobotic tasks were simulated, a 
pick-and-place task and three-axis tracking. Head movements were not needed 
to view the entire scene in these experiments. Also, the binocular disparity in 
the scene provided the only cue to die depth of the targets in space. The 
pick-and-place task required the operator to grasp objects that were floating 
within the workspace and put them in a box on the floor. Performance was 
measured by the average time to complete the task for 20 objects. In the 
three-axis tracking task, the operator followed a randomly moving target as 
closely as possible with a cursor controlled by two displacement joysticks, 
Performance was measured by the rms error between the target and the cur- 
sor. The error was normalized by the error value associated with no move- 
ment of the operator’s cursor. 


These tasks were performed under bioptic (binocular without disparity), 
ideal stereoscopic and blurred stereoscopic viewing conditions. The simulated 
se mes were drawn with lines and dots in order to reduce the number of com- 
putations needed to display and update the scene. Monocular target blur was 
induced by spectacle lenses and used in all the trials. This was done to ensure 
that a reduction in stereoacuity and performance occurred. Since disparity was 
the only depth cue present in the scene, performance was expected to 
deteriorate in a similar fashion to stereoacuity. Our results are directly com- 
parable to the psychophysical results since the method for inducing blur is the 
same, 


In Figure 7 (middle and bottom) the results show that performance of 
the pick-and-place and three-axis tracking tasks decreased when 2 diopters of 
monocular blur was present. The performance fell to the level of performance 
for the bioptic(B) viewing condition. Psychophysical results show that 
stereoacuity becomes 3-15 times worse with the addition of 2 diopters blur. 
This suggests that moderate reductions in stereoacuity can be enough to elim- 
inate the advantage of using stereo displays. This helps explain how early 
experiments comparing stereoscopic and monoscopic displays did not find any 
performance differences. [14] It is possible that the stereo displays were not 
caxefally adjusted such that the images seen by the operator was blurry. As a 
result, their stereoacuity was degraded to the point that the stereo display was 
not useful. 



Blur (Diopters) 

Figure 7. Blur Effects. The effect of spectacle blur on stereoacuity and tusk 
performance. Top: Data from Lit’s (square) and Westheimer and McKee’s 
(triangle) experiments. Middle: Me*, completion time for the pick-and-place 
task versus blur. The circle symbol is for an average of seven trials, solid 
circle is for one trial. Bottom: Error in tracking vs. blur. Wc examined the 
error along the x [horizontal] (cross), y [vertical] (plus), z [zoom] (diamond) 
axes and also in three dimensions (solid) 
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Previous Blur Studies 

Previous studies of blur as the stimulus to accommodation have been 
carried out by Dr. Stephen Phillips in our laboratory. Two aspects of the blur 
signal are of particular interest: — (i) the eccentricity of the stimulus with 
respect to the retinal fovea, and (ii) the spatial frequencies of the target before 
it is becomes the blurred retinal image. 

Two examples of the effect of eccentricity on the static and dynamic 
tracking ability of the accommodative system are shown in Figures 8 and 9. 
It is clear that the ability of a defocused peripheral target evoking an accom- 
modative response drops off rapidly outside the fovea; these results correlate 
with the decreased visual acuity. In normal viewing conditions vergence- 
accommodative synkinesis controls accommodation whenever a focysable tar- 
get is more than several degrees off the line-of- sight 

Spatial frequencies between 1.0 c/deg and 25 c/deg (at a 0.63 contrast 
ratio, Cm) are effective as shown in the experiment of Figure 10. Here, 
several subjects are focusing at a 5.5 D and 0 D distant target through a range 
of spatial frequencies. When the spatial frequency go beyond the feasible 
range the subjects drift back to their bias set level (generally about 1.0 
diopter, but for one of our subjects to about 2.0 diopter due to instrumental 
myopia). A summary figure , Figure 11, shows the envelope of responses to 
adequate and inadequate spatial frequency targets over a range of accommo- 
dative stimuli. 



Figure 8. Accommodation response as a function of eccentricity. A 4 s 
duration step stimulus at 4 D produced an excellent response at 0 deg eccen- 
tricity, but even a 1/2 deg eccentricity greatly reduced response. Response at 
2 deg is already likely only a searching stimulated by the subject noticing a 
change in light distribution. (Phillips, S. Thesis, 1974, University of Califor- 
nia, Berkeley). 



Figure 9. Accommodation Responses at Higher Eccentricities. In another 
subject, with excellent response at 0 deg eccentricity, little on no responses 
occurs at moderate eccentricities, (Phillips, S. Thesis, 1974, University of Cal- 
ifornia, Berkeley), 



SPATIAL FREQ (cycles/deg) 

Figure 10. Range of spatial frequencies serving as adequate stimulus to 
accommodation. Accommodative stimuli, containing frequencies from 1.0 
c/deg to 25 c/deg were adequate as shown in responses to 5 D and 0 D from 
a stimulus level of 1 D, for two subjects. Bias set level for one subject was 2 
D. (Phillips, S. Thesis, 1974, University of California, Berkeley). 



Figure 11. Envelopes of responses to adequate and inadequate stimuli. 
Accommodative stimuli containing spatial frequencies from 1.0 c/deg to 25 
c/deg were able to drive accommodation accurately over the range of diopters 
from 0 to 6 D (upper envelope). Stimuli containing only frequencies outside 
that range were inadequate (lower envelope). (Phillips, S. Thesis, 1974, 
University of California, Berkeley). 


Visual Search 

Visual scanning behavior, or how humans search their visual field, can 
be investigated through the use of the helmet- mounted display. These types 
of investigations of visual search would be useful in the development of 
electro-optical detection devices and manned systems designs. Much research 
has been done concerning visual scanning behavior using eye-movement data; 
however, head movements are also an interactive part of the visual scanning 
process, and is an area worth investigating. With the helmet-mounted 
display, viewing monitor windows limit the visual field (the area of extent of 
physical space visible to an eye in a given position) to a measurable amount, 
while the field of view (the extent of the object plane visible through, or 
imaged by an optical instrument) is digitally controllable. Thus, the subject is 
induced to use head movements with limited eye movements to search a 
given scene in a simulated field. 
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Figure 12 shows an example of head-movement data superimposed on 
the searched scene. A 3120 Silicon Graphics Iris was used to create and 
display this scene of robots on a simple background. Through the HMD 
monitors, only a windowed portion of the scene was visible to the subject at a 
time (see Figure 13). As the subject turned his/her head, a perspective pro- 
jection of the scene was displayed at an appropriate angle. The resulting 
effect was one of looking at a large two-dimensional wall. Figure 14 shows 
the actual display appearing on the HMD monitors while the subject viewed a 
central portion of the scene in fig 12. Here, the simulated dimensions used 
were of a 10 V* x 7 wall viewed from a distance of 2\ A 60-degree 
head rotation was required to scan the entire scene. 

Obviously, there are many factors which can affect visual scanning 
behavior. These include display resolution; number, orientation, and size of 
targets; amount of clutter (non- target objects resembling the target); field of 
view; noise; and peripheral visual acuity. Various eye movement studies on 
visual search have been conducted investigating the effect of the visual lobe 
area on visual search [10]-[12]. The visual lobe area is defined as the area 
surrounding the central fixation point from which information can be obtained 
in a single glance. As the complexity of the background in which the target 
is embedded is increased, the size of the visual lobe area decreases. Ulti- 
mately, the visual lobe area is a major factor influencing search time. This 
would be an interesting subject to investigate further using head-movement 
data. 



Figure 12. Head Movement Driven Visual Search. A typical search path 
(heavy line) with X's marking acquired targets, complete robots, in a field 
with clutter (partial robots) demonstrates that subjects do not make systematic 
searches. 



Figure 13. Visible Window Control by Head, Direction. To study visual 
search, we limited the solid window angle on a large display solid angle. 
This forced subjects to search the space with a combination of head and eye 
movements. 



Figure 14. The Visual Search Display. The HMD window provides only a 
limited view of the scene. In the figure the left image is inverted as in the 
actual display because the left viewfinder of the HMD is upsidedown. 
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